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Abstract. We develop basic constructions of the Baxter operator formalisni for the Macdon- 
ald polynomials associated with root systems of type A. Precisely we construct a dual pair of 
mutually commuting Baxter operators such that the Macdonald polynomials arc their common 
eigenfunctions. The dual pair of Baxter operators is closely related to the dual pair of recursive 
operators for Macdonald polynomials leading to various families of their integral representations. 
We also construct the Baxter operator formalism for the g-deformed 0Z£_,_]^-Whittaker functions 
and the Jack polynomials obtained by degenerations of the Macdonald polynomials associated 
with the type Ag root system. This note provides a generalization of our previous results on the 
Baxter operator formalism for the Whittaker functions. It was demonstrated previously that 
Baxter operator formalism for the Whittaker functions has deep connections with representa- 
tion theory. In particular the Baxter operators should be considered as elements of appropriate 
spherical Hecke algebras and their eigenvalues are identified with local Archimedean L-factors 
associated with admissible representations of reductive groups over R. We expect that the Bax- 
ter operator formalism for the Macdonald polynomials has an interpretation in representation 
theory of higher-dimensional arithmetic fields. 



Introduction 

A new class of operators acting on eigenfunctions of quantum integrable systems was introduced 
by Baxter to provide a solution of a class of integrable models |Ba] . These operators commute with 
quantum Hamiltonians of a quantum integrable system and satisfy difference/differential equations 
with coefficients expressed through quantum Hamiltonians. The Baxter operators were constructed 
for many integrable models including periodic Toda chains |PG| . 

In |GL01] we introduce Baxter operators for non-periodic 0[£_|_]^-Toda chains given by one- 
parameter families of integral operators. We also define dual Baxter operators acting on the spectral 
variables of Toda chain eigenfunctions. The dual pair of Baxter operators enters a canonical 
construction of a pair of recursive operators relating eigenfunctions of the fll^+i- and qI^- Toda 
chains. This gives rise to various families of integral representations for eigenfunctions. Hence the 
Baxter operator formalism consisting of a pair of dual Baxter operators, a pair of dual recursive 
operators provides a complete solution of the Toda chains. 

One can expect that the Baxter operator formalism can be constructed for a wide class of 
quantum integrable systems. Note that 0[^_|_]^-Toda chains can be considered as a degeneration of the 
quantum integrable system constructed by Ruijsenaars |Ruj and Macdonald [M]- The corresponding 
quantum Hamiltonians are given by mutually commuting difference operators and their common 
polynomial eigenfunctions are given by the Macdonald polynomials. In this note we construct 
Baxter operator formalism for the Macdonald-Ruijsenaars integrable system. This includes a dual 
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pair of Baxter operators, a dual pair of recursive operators and various families of explicit iterative 
expressions for the Macdonald polynomials, known and new ones. We also describe Baxter operator 
formalism for specializations of the Macdonald polynomials given by class one g-deformed gl^+i- 
Whittaker functions |GLQ2j and Jack's polynomials. Due to the results of |GL08j the Baxter 
operator formalism for the standard Whittaker functions [GL01| can be recovered from the Baxter 
operator formalism for (/-deformed gl^_|_i-Whittaker polynomials in the limit (7 — ?• 1. 

One should stress that the Baxter operators associated with the Whittaker functions have a 
surprising relation with number theory and representation theory |GL01] . Recall that the eigen- 
functions of gl^^^-Toda chains can be identified with particular matrix elements in the principal 
series representations of GL£+i(M) and thus providing generalizations of the classical Whittaker 
functions corresponding to SL2{^)- In [GL01| we argue that for g[£_,_|-Toda chain the Baxter oper- 
ator should be considered as a generating function of elements of spherical Hecke algebra associated 
with the maximal compact subgroup of GLi^i(M). Furthermore the 0[^_|_^-Whittaker functions are 
eigenfunctions of the Baxter operators with the eigenvalues given by local Archimedean L-factors 
of the corresponding principle series representations of GL^+i(M). 

This interpretation leads to establishing a deep relation between topological field theories and 
Archimedean algebraic geometry [GL05] . |GL06| . |GL07| . The construction of the Baxter operator 
formalism for the Macdonald- Ruijsenaars integrable system allows to define a {q, t)-generalization 
of the local Archimedean L-factors associated with principle series representations of GL^+i {q- 
generalization of local L-factors was introduced previously in [GL02]). One can expect that these 
generalized L-factors shall be related with principal series representations of loop groups associated 
with GLi^i. Taking into account the results of |GL05j . |GL06j . |GL07j one should look for a 
higher dimensional topological field theory interpretation of the Macdonald polynomials and the 
associated Baxter operator formalism. We are going to discuss this interpretation in the future 
publications. 

Finally we would like to point out that many of the constructions of this note are simple 
reformulations of the results of Macdonald [M] . However we feel that establishing the direct relation 
of the results of jM] with Baxter operator formalism might be useful. 

Acknowledgments: This work is partially supported by the RFBR grant 12-01-00894-a. AG 
was partly supported by Science Foundation Ireland grant. The research of AG and DL was also 
partially supported by QGM (Centre for Quantum Geometry of Moduli Spaces) funded by the 
Danish National Research Foundation. The research of SO was partially supported by P. Deligne's 
2004 Balzan Prize in Mathematics. 

1 Preliminaries on symmetric polynomials 

In this Section we collect basic facts on the Macdonald symmetric polynomials and their degenerate 
versions given by Jack polynomials and class one g- Whittaker functions. For details on Macdonald 
and Jack's polynomials see [M]; for class one g-deformed Whittaker functions see |GL02j . |GL03j . 
|GL04j . 



2 



1.1 Macdonald symmetric polynomials 

Let Q((7, t) be a field of rational functions in variables q and t. Define the following (q, t)-analog of 
the classical F-function (see Appendix for its basic properties): 

r,,(x) = , (x; g)oo = - ^9')- (1-1) 

[x, q)oo ^.^g 

Let Aq^t be the graded Q((7, t)-algebra of symmetric polynomials of variables xi,X2, ... of degree 
one 

^9:* = ' (L2) 

n>0 

where A^"^"* is the homogeneous component of Ag j of degree n. There are various convenient 
bases in the space of symmetric polynomials in variables xi,...,X£+i enumerated by partitions 
A = (Ai > . . . > Af+i), Aj € Particularly, the elements of the bases of monomial symmetric 
functions m\{x) are given by sums of all distinct monomials obtained from = x^^ . . . x^^-^ by 
permutations of Let us denote Pn{x) := the symmetric polynomial for the 

partition (n) = (n,0, ...,0). The bases of power series symmetric polynomials consists of the 
polynomials px{x) = p\^{x) ■ . . . ■p\f_^-^{x). Equip the space A^^^^^ with a scalar product ( , ) defined 
by 

{p\,Pf^)q,t = hfi ^>^Y\. I _ fX, ' ^ JJn™"m„!, m„ = \{k : Afc = n}| . (1.3) 

i=l ' n>l 

Macdonald introduced a bases {P\{x) = P\{x] q,t)} of symmetric polynomials over Q{q,t) enu- 
merated by partitions A such that 



P\{x) = ^ u\t^ m^, uaa = 1 , 

^l<x 



(L4) 



and 

{Px, P^)g,t = 0, A/^. (L5) 

In the above formula < denotes the natural ordering: 

X < ^l <^=^ Ai + . . . + Aj < fii + . . . + fXi , i >0. 

The relation ()1.4p is invertible and thus the Macdonald polynomials P\{x) provide a bases in 
^(£+1)^ The inverse norms of the Macdonald polynomials are given by 

where the product is over the boxes in the Young diagram attached to partition A, and A""" 
denotes the conjugate partition. In particular, one has 



n 



_ _ r,,,^.(g) 
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The Macdonald polynomials Px{x) can be also characterized as common eigenfunctions of the 
following set of mutually commuting difference operators [M]j |Ru] acting in A^^^^^ 

where the sum goes over all r-element subsets Ir of (1,2,...,^ + !) and 

The operators Mr, r = 1, . . . , ^ + 1 are self-adjoint with respect to the scalar product (|1.3p and the 
eigenvalues of Mr acting on Px{x) are given by the elementary symmetric functions 

Xr{yi,---,ye+i) = ^ yh-----yir, yi = t^'q^\ (^i_9) 

l<il<...<ir<^+l 

where Qi = i + 1 — i. The eigenfunction property of the Macdonald polynomials can be succinctly 
described by the relation 

Me+i{X) ■ Px{x) = Q+i(A; X) Px{x) , 

^+1 (1.10) 



.i(A; X) = (1 -t)-(^+^)]J(l + t^'»g^»X; 



i=l 



where A4i-^-i{X) is a generating function of the difference operators (jl.Sp : 
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M^+i{X) = {l-t)-^'+^Ul + Y,X^Mr). (1.11 



r=l 



Assume now that g G C and \q\ < 1, so that infinite product (z; q)^^ converges for all z € 
Following Macdonald define a new scalar product 

(O' b)'t = / d^z a(z)6(z-i) A(z; q,t) , 



+ 1)! 



T 



e+1 ^ (1.12) 
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where a{z) and b{z) are Laurent polynomials and 

T = {z= (zi,...,Z£+i) : |zi| = l, i = l,...,^+l} 

is the (^+l)-dimensional torus, with the Haar measure z = Y[i=i(.'^'^'^)~^'^^'^S ^i- The polynomials 
Px are pairwise orthogonal with respect to the new scalar product (, )' with the norms given by 



i<j 



Let us recall the properties of the Macdonald polynomials that will play essential role in the 
following (see [M] for the proofs). 
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Theorem 1.1 Consider two sets x = (xi, . . . , Xn) and y = (yi, . . . , y^) of variables. Let 



X&Yr 



Ti.m 



where summation goes over a set Ym^n of the partitions of length min(m, n) and h\ are given by 
()1.6p . Then the following identity holds 

n m 

Un,m{x,y) = YlYl'^qA^iVj)- (1.15) 

i=lj=l 

We will say that partitions /i, A are interlaced if /ii > Ai > . . . > fii^i > Xe+i- In the sequel we 
shall use the following abbreviation for interlaced partitions: fJ-i > Xi > fJ-i+i- 

Theorem 1.2 Let be the Macdonald polynomial corresponding to the partition (n) = 

(n, 0, . . . , 0). Then the following product decomposition holds: 

P{n){x)xPx{x) = b^^^ J2 V^,/xP^.{x), 

I /i I — I A I — n 

where 

IX r,,*.- {p-r'^^') r,,,-i (t.-QA.-,,+,+i) ' (1.17) 



',3 



and we omit in the product ()1.17p the factors depending on A^+2 and 

The Macdonald polynomials possess a remarkable self-duality property discovered by Koorn- 
winder (see [M] and references therein). Let us introduce modified Macdonald polynomials: 

i+i 

^x{x; q,t) := r,,t(t^-^^"-^^) x Px{x; q,t) , (i.ig) 



a<b 



e+1 

where p{x) = ^ PiXi, pi = Qi —i/2. Then for any partitions A and p the following duality relation 

i=l 

holds: 

'^xiq"-''; q, q-") = ^^(9^"'"; 9, 9^') ■ (1-19) 

This duality naturally leads to the set of mutually commuting operators acting in the space of 
functions on the set of partitions A = (Ai, . . . , A^+i) G Z^^. The following Theorem was proved in 
[GL04j . 

Theorem 1.3 ( |GL04] ) A set of mutually commuting difference operators 



1 J.?— 7 + 1 — A, — 1 1 j-i—j — l^X-i—Xi 

7{x) = f'/'y IT -l X \ ■ X X Tr , 



f<r (1.20) 



'Pq,q^i ■ /(Ai,..,A«+i) — /{Ai,...,Ai+l,...,Af+i) ) ~ 11 

ieir 



q, q^i ' 
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acts in the space of functions fx labeled by partitions A = (Ai > . . . > A^+i) G Z^^. The Macdonald 
polynomials Px{x) as functions of the variables A are common eigenf unctions of the difference 
operators (jl.20p ; 

X)Vi(X) • Px{x; q,t) = c,V(t^/2x; X) Px{x; q,t) , 

^+1 (1.21) 
cy+i(x; X) = l[{l + Xxi), 

i=l 

where 

e+i 

Mi^,{X) =Y.X^ , := 1 , (1.22) 

r=0 

1.2 Class one g-deformed Whittaker functions 

Let Aq be the algebra of symmetric functions in variables xi,X2,--- over the field of rational 
functions in variable q convergent in the domain \q\ < 1. Let Ag^"*"^^ be the homogeneous component 
Aq of degree i + 1. Consider a pair of the scalar products on A^'^^\ The first one is defined in 
terms of power series symmetric polynomials p\ as follows: 

e+i 

{px,P^.)g = Sx^^zxHil-q^^). (1.23) 

i=l 

The second scalar product on the space Ag is defined by 

(«(^)' K^)); = 77^ f d^'z a{z)b(z-')A^Jz), 



+ 1 ! 



T 



e+1 oo (1-24) 

A^z) = rr L , r(x) = n — 



^gi^i ^j) j=o 



and the notations of (|1.12p are used. 

Lemma 1.1 The polynomials Px{x;q) := Px{x;q,t = 0) satisfy the relations 



Pxi^) = '^UXfj.mfj,, MAA = 1 



(1.25) 

Px, P,.)q,t = 0, A///. (1.26) 



and thus define a bases in A, 



q- 



Proof. Directly follows by specialization t = from the properties of the Macdonald polynomials. 

□ 

Let us define the normalized symmetric polynomials in variables xi, . . . , x^^i, labeled by parti- 
tions A = (Ai > . . . > A^+i) G as follows 

e 

Pf{x) =Aq\X)Px{x;q,t = 0), A,(A) = JJiK - Xi+i)ql , (1.27) 
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where (n)q! = Iir=i(l — Q^~^^ In the following we will call (x) the (7-Whittaker polynomials. 
These polynomials were introduced in |GL02] as class one g-deformed gt^^^-Whittaker functions 

The (7-Whittaker polynomials P^^(x) are orthogonal with respect to both scalar products 
(jl.23p . (jl.24p and are normalized as follows: 

Theorem 1.4 ( |GL04] ) Let Hi, ... , Hi+i, and , . . . , H^_^_l be difference operators acting in 
the space of functions on M.^^^ x 



= E 11(1 - T/^ , Tl = ll r^,^.. , (1.30) 

Ir k=l iGlr 

for r = 1, ...,£+ 1, where ir+i := i + 2 is assumed. 

These operators are mutually commutative and the q-Whittaker polynomials solve the following 
dual pair of eigenfunction problems: 

■ Pf(x) = q^t+2-r+-+^e+i pf{x), r = l,...,i + l. (1.31) 

and 

Hr ■ Pf^ (x) = Xr{x) Pf^ (x) , Xr (x) = ^ ■ ■ ■ Xj^ , 



:i.32) 



for r = 1, ...,£+ 1. 



For the generating function -D^+i of the operators H^. 



D,+i{X) = Y^X'Hr , ^0 := 1 , (1-33) 

r=0 



the following relation holds 

Z),+i(X) . Pf{x) = c\^,{x- X) Pf{x) , 

^+1 (1.34) 
c,%,(x; X) = l[{l+Xxi). 

i=l 

The set of operators (|1.30p define g-deformed Toda chain Hamiltonians and (|1.29p provide a set 
of mutually commuting difference dual Toda chain Hamiltonians introduced in [GLU4j . 
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Using the relation between g-Whittaker polynomials P^^(x) and Macdonald polynomials P\{x) 
one can infer an analog of the Fieri formula (I1.16P 



— I A I — Ji 



:i.35) 



e(/il - Al) f-r @{Xi - ;Ui+l) 0(/ii+l - Xi+l) 



The analog of the Cauchy-Littlewood identity (jl.l4p is given by 

n m 

i=lj = l \&Yn,m 



:i.36) 



summed over A = (Ai > . . . > Am) with m < n. 

Remark 1.1 The q-Whittaker polynomials P^^(x) can be also obtained from the Macdonald poly- 
nomials P\{x) under the limit t = , k — ?• +oo. Let 



D{x) = n 



kee+2- 



1=1 



i=l i,j=l " ij,<? J q \q 

Then we have 



Pf'{q^)= lim Z)^(A)-iZ)(#+'=^')-i xPx+fc^CgP+^'C^+i)) . (1.37) 
— fc— i>+oo L J 

for p = {pi, . . . ,pe+i) be a partition {pi > ■ ■ ■ > P£+i), and Pp^ (q^) = otherwise. 
1.3 Jack's symmetric polynomials 

Now we consider a bases of symmetric polynomials consisting of the Jack polynomials, obtained 
from the Macdonald polynomials by a specialization (see [M] and references therein). 

Let K be a positive integer, and let A^^^^^ be the homogeneous component of degree £ + 1 in 
the algebra of symmetric functions in variables xi,X2 ■ ■ ■ over Q{k) = Q. Define a pair of scalar 
products on A^^"*"^) using the standard bases {px} of power series symmetric polynomials: 

(pa, P^.). = ^"'^^^5a^ zx , /(A) = \{m\ A„ / 0}| , (1.38) 

where -?a = H n"^"rnnl and = |{/^ : Afe = n}|. 



n>l 



{px,Pt.y. = TT^yyy jj^'z pxiz)p^iz ')A(,)(z), (1.39) 



T 
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where 



^(«)(^) = n ~ ^i^^jY ■ (1.40) 



iit] 

Definition 1.1 Jack's symmetric functions P^f^ are the elements of A^^'^^^ such that 
whenever \^ ji, and 

P'^' = mx + ny . 

fj.<\ 

The Jack polynomials P^**^ (x) are orthogonal with respect to both scalar products and the following 
normalization condition holds: 

/pW pWv = fr r(A, - Xj + Kjj -i + l)) TjXj - A,- + 1 + K{j 

^ ' ' " r(A,-A, + K(i-z)) r(A,-A, + i + K(i-z)) • (1-41) 

Similarly to the cases of the Macdonald polynomials and g'-Whittaker polynomials, Jack's poly- 
nomials are eigenfunction of dual families of mutually commuting differential/difference operators. 



Theorem 1.5 (i) The Jack symmetric polynomials are eigenf unctions of a set of mutually com- 
muting Sekiguchi differential operators: 



• P^^\x) = \{[X + (A, + ^.At)) (x) , (1.42) 



i=l 

where 

r=l 



i<3 



(1.43) 



with %Q := 1. 

(ii) The Jack polynomials are eigenf unctions of a set of mutually commuting difference operators 

P^,)(X) .Pf (x) = c,V(x; X)pf (x), 

e+i (1.44) 
c)Vi(x; X) = l[{l + Xxi), 

i=l 

where 

e+i 



r=0 
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and 



^'rW = E n 



; - j + 1)k + \j - \i - I 
{i - j)K + \j - \i - I 



(1.45) 



Here the summation goes over all r-element subsets Ir of {1,2, ... 1) and the operators are 
defined as follows 



q,q^ 



Tq^ gA • fx 



ieir 



fx+5i j , Ai_i > Ai + 1 







otherwise 



where X + 5ij := A + (0 



0) 



Remark 1.2 The generating function ()1.43p can be considered as an appropriately defined non- 
commutative determinant: 



1 



det \\x 



det 



/ d 

+ QjK + Xi-Q^ 



In particular, the first statement implies: 

r = l,...,£ + l 

where Qi = £ -\- 1 — i . The first two Hamiltonians are given by 



(1.46) 



d 

i=i * 



i=l 



(1.47) 



The Jack symmetric functions can be obtained from Macdonald polynomials by taking the limit 



lim Vq^t{x) 



(1-x)- r(n)r(/^)' 



t = e' 



e\ 



(1.48) 



It is easy to infer analogs of p.l6|) and p.l4|) for Jack polynomials. In particular, the Fieri rules 
for the Jack polynomials are given by (see [S]): 



(m) A (k) 



(m) Mi>Ai>Hi+i 

I ^1 — I A I — n 

(«) _ TT r r(^» - + 1 + {j - i)K) r(/i, - Xj + {j -i + i)k) 
'^^/^ ~ ^ r(/i, - /X,- + U-i + 1)'^) r(^. - A, + 1 + (j - i)K) 

T[\i - Aj+i + 1 + (j - z)k) r(Aj - /ij+i + (i - i + 1)k) 



(1.49) 



r(Ai - X.j+i + {j-i + 1)k) T[Xi - fij+i + 1 + (j 



IK 
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and in the product (jl.49p we omit the terms containing Xe+2 and The analog of the Cauchy- 

Littlewood identity (I1.14p is given by 

nil -11(13^= E b[-'pt\-)Pt\y), (1.50) 

,m 

where the summation goes over partitions A = (Ai > . . . > )^rtiin{m,n)) and 

(k-\ -i-r i^i^J + 1 — i) + Aj — j ^ f- 

bt^ = limb,= n XT -^^ ^ ' * = ^ ' « = ^- (1-51) 



2 Baxter operator formalism for symmetric polynomials 

In the previous Section we describe various bases in the space of symmetric polynomials defined 
as common eigenfunctions of two sets of mutually commuting operators called (dual) quantum 
Hamiltonians. In this Section we define a dual pair of the Baxter operators acting in the space 
of symmetric polynomials, commuting with dual pairs of quantum Hamiltonians. The constructed 
bases in the space of polynomials is also a bases of eigenfunctions of the dual pair of Baxter 
operators. 

2.1 Baxter operator formalism for Macdonald symmetric polynomials 

In this Section we develop the Baxter operator formalism for the Macdonald polynomials. We 
construct a dual pair of Baxter operators and a dual pair of recursive operators. This results in 
various families of integral/sum representations for the Macdonald polynomials. 

Definition 2.1 Baxter operator = Q'^/{q,t) associated with Macdonald integrahle system is a 
family of operators acting on the space A^^^^^ of symmetric polynomials as follows: 

Q7 • P{x) = d^'y Q^{x, y) A(y) P{y-^) , 7 G Z , (2.1) 
where integral kernel is given by 

e+i e+i 

Qj{x,y) = Y{{xiyiy rq^tixiyj). (2.2) 

i=l *J=1 

Theorem 2.1 The Baxter operator ()2.ip acts on the Macdonald polynomials P\{x) as follows: 

■ Pxix) = L^(A) Pxix) , Xe+i > 7 (2.3) 

Q^-Px{x) =0, 

where 

L^(A) := L^{X,q,t) 



Xi+i < 7 



(2.4) 



e+i 

n— 

i=i 



q,tq- 



(2.5) 
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Proof. The Baxter operator (j2.ip can be represented in the fohowing form 

:= D?!'+' o C0'^+i o , (2.6) 

where the operator C^'^^+i acts as: 

C0'^+i • Px{x) := (n,+i,,+i, PA>^,i , (2.7) 

and the operator 1)^'^+^ in ()2.6p acts on Px{x) according the fohowing rule: 

• Px{xi, . . .,xe+i) := Px+(e+ip{xi, . . .,xe+i) 

(2.8) 

= {xi ■ . . . ■ xe+i)'^ Pxixi,. . . ,xi+i) , 

Px+{e+i)-i (x) := P\i+7,...,Af+i+7(2;). The eigenvalue of the operator C^'^+i can be found explicitly 
using the Cauchy-Littlewood identity (jl.l4p and orthogonality of the Macdonald polynomials with 
respect to the two scalar products, ()1.3p and ()1.12p 

CsWi.pf+i(x) = ^dXyn,+i,,+i(x,y)pf+Hy-i)A(y) 
= /d^y( E b.^^^"Hx)P^^-(y))pf-(y-^)A(y) 

= E ( / ciXyP^^'^+Hy)^f^"^(y-')A(y))p^^+Hx) 



1 i=l 



e ipBk+i pSh+iy 



Therefore the eigenvalue of the dual Baxter operator on Px{x) is given by 

e(A^+i - 7)— ^ — = e(A^+i - 7)- 



(-fA-(^+i)^) P\-{t+i)-i)q,t {P\-{t+iYi^ Px-(i+i)y)q,t 

= bx-{e+ip&(.>^e+i - l){Px, P\)'q,t ■ 
One can rewrite the right hand side of (II. 6p in the following form: 



r„,„-i(i^-V'-^.+i) 

= iife(A.-A.+i) X [[ ' , j_, x,-x,+, + iy ^^+2 :=0' (2.10) 



where 

1 - ig'^ 
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1=1 



Then combining (j2.10p with ()1.13p one readily arrives at (j2.3p . □ 

There is an analog of the classical Baxter equation (see Theorem 2.3 in |GLUlj ) relating Baxter 
operator and quantum Hamiltonian operators. 
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Proposition 2.1 The operators given by (|2.ip and the generating function A^^_|-i(X) from 
(jl.lip commute and satisfy the following relation: 

M,+i{-q~^)oQ^{q,q-^) = Q^^^{q,q^-^) . (2.11) 

Proof. Recall that the Macdonald polynomials are common eigenfunctions of and M.iJ^i{X). 
Thus it is enough to check ()2.1ip on common eigenvalues of A^^+i and acting on P\{x). Denoting 
L-y(A) and q_|_i(A) the corresponding eigenvalues: 

t+i l+i 
L^{\ + kg- q, q-^) = J] &{A.-7) , + kg-^ -q-^) = (1 - t)-^'^'^l[{l - (?^-^) , 

1=1 i=l 

we easily check the following relation: 

ce+i{X + kQ; -q"^) X L^{X + kg; q,t) = L^+i{X + kg ; q,tq) . (2-12) 

This entails the operator relation (j2.1ip . □ 
Now we define the dual Baxter operator. 

Definition 2.2 The dual Baxter operator = ^z{q,t) is a family of operators acting in A.^^^^^ 

^z-P\{x)= ^ (^^(A, /i)P^(x) .2.13) 

with the kernel function 

4(A, ^) = >I-I^I(P;./A , (2.14) 



and 



(2.15) 



:Q{fii - Ai) e(Ai - /ii+i)e(^i+i - Ai+i) , 



i=l 

where in the product one should omit the factors depending on A^+2 o-nd 

Theorem 2.2 The action of the dual Baxter operator on the Macdonald polynomials reads 

4-Pa(x) = Lr(^)A(x), (2.16) 

where the eigenvalue is given by 

i+i 

Li{x) = l[Tg,t{zx,). (2.17) 

i=l 
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Proof. The statement of the Theorem directly follows from the Fieri formula for the Macdonald 
polynomials (fTTG]) . (fTTTD : 

A'i>Ai>/ii+i 
oo oo 

= E^" E V^M/A^A.(^) = (E (2.18) 

m=0 Aii>'^i>Aii + l 771=0 

\fj.\ — \X\ —m 

= Ui+i^i{x,z) Px{x) , 
and identification L^{x) = n£+i^i(x,z). □ 

Proposition 2.2 T/ie operator satisfies the following difference relation: 

Mt^,{-q^'/'z)o^,{q,q-^) = ^,.{q, q-'-') . (2.19) 
where A4'^_^_-^^{X) is the generating function (jl.22p of the dual quantum Hamiltonians. 

Proof. It is enough to check (|2.19p on common eigenfunctions P\{x) of A^^+i and Q)^ . Denoting 
Vjj^ and c^^-^ the corresponding eigenvalues 

L^zix) = YlVg^tizx^) , cy+iCx; -t-^/^z) = J](l - zxi) , 
1=1 i=l 

we easily check the following relation: 

c,V(x; -t-'/^z) X L^Ax;q,t) = q,{x ■ q,tq-^) . (2.20) 

This entails the operator relation (j2.19p . □ 
Let us introduce the following notation 

—n — ('^n,l ) • • • 1 O-n, n) i Qlu — (^n,! j • • • ; Ojn, n— l) • 

The following recursive relations hold; the first one (see [AOS] ) 

P\{xi+i) = J d''xeQf^'+'{xi_^-^■, x^\ Xi+i) A{xe) Py{xJ^) , 

I (2.21) 

i=\ 

and the dual recursive relation (see [M]): 

^A,^i(^) = E^;ir(A.+i, a,ix,+i)pa,(x'), 

(2.22) 

where 

T-r r,,,,-i(t^--g^'-/-^+i)r,,,,-i(^^--g^'-^^+i+i) 

ii<i<£ r,,*,-i(i^-^9'-'^^+^) r,,,,-i(ii-g'^-^^+^+i) ^2-2^^ 
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when A and /i are interlaced (i.e. Ai > /.ii > . . . > > /i^ > A^+i > 0), and V'A/^t = otherwise. 

These recursive relations allow to introduce the corresponding recursive operators Qgl""*"^ (A^-f 
and^;;;;+nxn+i): 



QSr^A.+i) • /fe+i) = _^d><x,Q;;;+i(x,+i;x,|A,+i)A(x,)/(x7i), (2.24) 

and 

^tiTi^i+i) ■ /(A.+i) = E^Sr'(A^+i, A,| xe+i) fix,) , (2.25) 

The existence of the two dual recursive representations, (|2.2ip and (j2.22p . provide a family of 2^ 
integral representations for the Macdonald polynomials. Namely, let us change our notations as 
follows: 

^i+l,n ■= Qgll'^^i^n+l) , ^n+l,n ■= ^fll'^^ i^n+l) , n = l,...,£; 

then for every array e = (ei, . . . , e^) of e„ G {/, //}, n = 1, . . . ,£ the following holds: 

= { R7+i,i° • • • ° ^2,1 ° «i,o} • 1 , (2.26) 

where -Ri^o is the gl^-Macdonald polynomial P^'i. 

Let us remark that the recursive operators can be factorized into Baxter operators, similarly 
to recursive operators for g[£_,_x-Whittaker function (see Proposition 3.3 in |GL01j ). This reveals 
the fundamental role of the Baxter operators in the description of various bases of symmetric 
polynomials. 

2.2 Baxter operator formalism for class one g-Whittaker functions 

Now we provide similar results for g-Whittaker polynomials P^^(x). 

Definition 2.3 The Baxter operator acting in Ag^"*""*^^ is a family of integral operators 

Qz-fW= Agin) Qi+ij+i{fx; X\z) fin) , (2.27) 

with the kernel 



Q,+i,,+i(^, A| z) = zl^-^l ^l^^ = zl^'-^l ^^/a((?, t = 0) X Ag(A)- 
-A| ©(/^i - ■^i) TT ®(^« ~ /^i+i) 0(mi - Aj+i) 



(/^i - Ai),;! n i^x- _ ^i+i)g! (;xi+i - Ai+i) 



(2.28) 



Theorem 2.3 (i) The action of the Baxter operator on q-Whittaker polynomials (jl.37p is given 
by 

Gt,-Pf{x) = L,{x)Pf'{x), (2.29) 
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where 

L,{x) = YlVgizxi) . (2.30) 

i=l 

(ii) The operators and D'^^^{X) (|1.34p satisfy the following relation: 

De+i{-z)oQ, = Qg,. (2.31) 

Proof. The relation p.29p is a direct consequence of the Fieri formula ()1.35p . The relation p.3ip 
follows from the relation between the corresponding eigenvalues: c^_|_^(x; —z)'Lz{x) = Ijqz{x). □ 

Definition 2.4 The dual Baxter operator acting in A^'^^^ is a family of integral operators 

^^■P{x)=l^d''yQ^,^,j^,{x,y;j)A^g{y)P{y-^), 7EZ, (2.32) 

with the kernel 

e+i 

^e+i,e+i{x, y; 7) = Y{ {xiyiyTg{xiyj) . (2.33) 
«,i=i 

Theorem 2.4 (i) The action of the dual Baxter operator on q-Whittaker polynomials reads as 
follows: 

(k,-Pf{x) = mX)Pf{x), LV(Ai,...,A,+i) = ^ (2.34) 

when 7 < A^+i, and 

^^■Pf{x) = Q, 7>A,+i- (2-35) 

(ii) The dual Baxter operator satisfies the following difference equation: 

[1 - q-^ii(^o^^ =^^^i . (2.36) 

Proof. The first statement follows from (jl.36p and the orthogonality of the g-Whittaker polyno- 
mials (see |GL01| ). The second statement follows from the relation between the corresponding 
eigenvalues: (1 - q^^+^-y) x L:^(A) = L^+i{X) • □ 

In |GLQ2] and |GL04| the following recursive relations for the g-deformed Whittaker functions 
were established. 

Proposition 2.3 ( |GL02) . [GL04] ) The following recursive relations hold: 

Pf{^) = (QSr(^m)-^"'^(^')), 
= Yl Qm,£(A;^|x,+i)A,(/.)P^«^(x), ^^-^^^ 
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where 
and 

Pfi^) = ^glr^A.+i) • Pf{x) = j^d-y^,-,iA^; y\ A,+i) ^^{y) Pf{y-') . (2.39) 

where 

^e+ij{xi,. . . ,xe+i; yi,- ■ ■ ,ye\i) = '[lx]Y{y] n ^li^iyj)- (2.40) 

i=l 7 = 1 i<i<e+i 
i<i<i 

The action of recursive operators (|2.37p and ()2.39p provide a pair of dual integral/sum represen- 
tations of the (/-deformed Whittaker functions (see [GL02) and |GL04| ) . Combining the recursive 
operators of different types one can obtain 2^ exphcit formulas for Whittaker functions similarly 
to ([2:26]) . 

Remark 2.1 The recursive operators QgJ'^^^ (x^+i) and (^^j^^^ (A^+i) can be factorized into the Bax- 
ter operators (|2.27p and (j2.32p , similarly to Proposition 3. 3 from IGLOl^ . 

2.3 Baxter operator formalism for Jack's symmetric polynomials 

Now we consider Baxter operator formalism associated with the Jack symmetric polynomials. 
Definition 2.5 Baxter operator is a family of integral operators acting in A^^"*"^^ by 

QW-p{'^)(x)= /d>^yQW(x,y;7)A(,)(y)p{'^)(y-i), 7GZ, (2.41) 

JT 

with the kernel 

t+i 

Q('^)(x,y; 7) = \{{x,yiVlif^^j^^{x,y) . (2.42) 
1=1 

Theorem 2.5 (i) The action of the Baxter operator Qi^^^ on the Jack polynomials is given by 

when 7 < A^+i + k, with Qi = £ + 1 — i,i = l,...,l+l, and 

Q('')-P{'')(y) = 0, ^>A,+i + At. (2.44) 

(a) The Baxter operator Q^f^^ commutes with the generating function P^+i (11.431) and satisfies 
the following difference equation: 

P^+i(/^-7)oP,+i(l-7)"'°Qi"^ = Q^^li- (2-45) 
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Proof. (I2.43P follows from (jl.50p and the orthogonality of the Jack polynomials [Ml • ^ 



Definition 2.6 The dual Baxter operator ^i'^^ is a family of operators acting in A^^^-*^) by 

^W.pf\x)= 5] (^W(/x,A;z)pW(x), (2.46) 



with the kernel 



Aw^y X. = zM-\M fr r r(M» + i + (j - ^)^) r(^^ - + ij-i + i)k) 

^ ^ ^ .ii ^T{fi,-fi, + {j-i + l)K)T {fii - A,- + 1 + (j - i)K) 

i<j 

T[Xi - Aj+i + 1 + (j - r(Ai - Hj+i + ij -i + 1)k) 



(2.47) 



r(Ai - + (j -i + 1)k) r(Ai - fij+i + 1 + {j - i)K) 

xe(/ii - Ai) Jl e(Ai - ^i+i) e(^i+i - a^+i) . 

i=l 

Theorem 2.6 (i) The action of the dual Baxter operator on the Jack polynomials is given by 

^(")-Pf (x) = Z,(x)Pf (x), (2.48) 

where 

(ii) The dual Baxter operator commutes with given by (jl.44p and satisfies the following dif- 
ference equation: 

Proof. The first relation (j2.48p follows from the Fieri formula (jl.49p . and the Cauchy-Littlewood 

identity (11.50p is implied by the relation between the eigenvalues: c]^j^i{x; —z)x/ii^\x) = 
□ 

The recursive relations ()2.2ip and (j2.22p imply similar recursive relations for Jack's symmetric 
functions. Namely, the following recursive relation hold (see [AMOSl] , [AM0S2] . [AOS] ): 



/ d'^Xi Q^+i ^(x^+i; x^l A^+i,i;+i) A(^)(x^)P^r^ {xj'^] 
Jt ' -'^+^ 



(2.51) 



where 



Qe+ij{x£+i] Xi\ A^+i,£+i) 



^+1 ^ 1 (2.52) 

xe+ij+i ^^^^ , , . 

* *i J ) 
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We also have the dual recursive relations 



P. 



W/ / X (2-53) 



Here 



'3h 



^(/i; X\z) 



n[ 



r(/xi - /ij + 1 + (j - i)K) T{ni - Xj + {j -i + 1)k) 



I.J 

i<j 



J-^ L r(//i - + (i - i + i)k) r(;Ui -Xj + i + {j - i)K) (2.54) 

r(Ai - Aj+i + 1 + (j - i)K) r(Ai - + (i - i + 1)k) 
r(Ai - Aj+i + (i - i + 1)k) r(Aj - + 1 + (j - i)K) 



when (//I, . . . , /u^+i) and (Ai, . . . , A^, 0) are interlaced, and Ogi*^^ (m; A| z) = otherwise. Obviously 
for Jack polynomials one has the proper analogs of the mixed integral/sum representations ()2.26p 
of the Macdonald polynomials. 

Example 2.1 The simplest dual recursive operator intertwining and gli Jack's symmetric func- 
tions reads as follows: 



(2.55) 



(^g[j(A2i, A22; All I X2) 

^ 2,A2i+A22-An r(K + A21 - All) r(K + All - A22) (A21 - A22)! 

2 r(K + A2i-A22) (A21 -Aii)!(Aii - A22)! ■ 

This leads to the following representation of the QI2- Jack's polynomial: 

pW ^ A - V" r(At + Ai -^)r(At + /i- A2) (A1-A2)! „ A1+A2-/. 

n„A2(^i'^2j - 2^ r(K + Ai-A2) (Ai-m)!(m-A2)!^^'^^ 

Remark 2.2 The recursive operators QgJ^^^(A^_|.i^£+i) and ^^j^^^ (x^+i^^+i) can he factorized into 
the Baxter operators (|2.4ip . (j2.46p , similarly to Proposition 3.3 from \GL01^ . 



3 Appendix: Various analogs of classical F-function 

In this Appendix we provide basic facts on the analogs of classical F-function arising in the Baxter 
operator formalism for Macdonald, g-Whittaker and Jack polynomials. 

Classical T-function can be defined by analytic continuation of the function defined by Euler's 
integral representation: 

r(s) = fdt e'^e-^' , Re(s) > 0. (3.1) 

Jr 

Equivalently F-function is defined as a solution of the functional equation 

r(s + i) = sr(s), r(i) = i, 
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such that is an entire function on the complex plane. F-function allows a representation as the 
Weierstrass product 



n=l 



r(l + s) = e-^^ J] (l + ^) \ (3.2) 



where 7 = — r'(l) is the Euler constant. Also the following reflection property holds 

r{s)xr{i-s) = (3.3) 



(3.4) 



Note that the integral representation (|3.ip can be inverted via the Mellin transform 

2711 J ^ ' 

Define {q, t)-analog of the classical F-function as the following infinite product 

= n ' (3-5) 

n=0 ^ 

where we imply that q is variable taking values in \q\ < 1. This function has poles at x = q~"^, 
m € Z>o and zeroes at x = t~^q~'^, m € Z>o. 

The function F^^^ defined by p.Sp possesses all the basic properties of the classical F-function 
outlined above. The analog of the Weierstrass product is given by (|3.5|) . The analog of the relation 
p.3p is given by: 

F,,,(.) X F,,,_.(g.-^) = ,1/2 (3.6) 

Oi[z^/^; q) 

where we take into account the product representation 

ei{z; q) = q^l^ - q^){l - z\^){l - z'^^) . (3.7) 

' i>l 

of the standard elliptic theta-function 6i{z; q). 

Finally, the analog of the Euler integral representation (j3.ip and its inverse p.4p are given by 



i.,Ux) - r (^.+1), F,.,-.(,A+1) - 7/ 



Consider now a specialization of the Tq^t at t = given by 



(3.8) 



1 00 ^ 

W:=^ = nr^- (3.0) 

The (7-Gamma function Tq{z) has poles at z = q~"^, m E Z^. and satisfy proper analogs of ()3.ip - 
p.3p . The (/-analog of the Weierstrass product formula (j3.2p is given by (j3.9p . The g-analog of the 
Euler integral formula (13. ip is given by 



w-E;^f^. f^-//--^^.(=)' (3.10) 
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and the g-analog of the functional equation ()3.3p has the fohowing form: 

,1/4 ^^-1/2 



r,Wxr,(,=-') = A___. (3.11) 



which can be deduced from (|3.7p . 

Now consider the following analog of the F-function 

rW(z) = hm r,,,(x) = (j^y, t = q = e^ (3.12) 



depending on a positive integer parameter k. The analog of the functional equation (j3.3p reads 

rW(z)xr(-«)(z-i) = (3.13) 

and the binomial formula for r^'^\z) = {l — z)~'^ implies the following analogs of the Euler's integral 
formula (j3.ip and its inverse: 

The functions Tq^t{x), ^q{x) and r('')(x) play an important role in the Baxter operator formalism 
for Macdonald, Q'-Whittaker and Jack polynomials correspondingly. 
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